1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)? Goes against Commandment #2, Though shall not employ ambiguous statements in your test item.
Evidence: Commandment 2: The word, 'they', appears in the questions. Due to the way the question is worded, we can't tell who 'they' is referring to, the administrators or the teachers?? This will affect how one may interpret the question, and therefore it flaws the question as a whole.
Binary Choice Item # 2 - flaw(s)? Goes against Commandment # 4, Though shall not employ complex syntax AND Commandment # 5, Though shall not use vocabulary that is too advanced for the audience.
Evidence: Binary Choice Item #2 uses extremely advanced and complex vocabulary that makes it difficult to comprehend what the question is even saying. Our whole group agreed that it was pretty much impossible to get through the whole question and actually understand what was being asked. This makes the question too difficult to answer. Commandment #4 also states to construct simple sentences. This question item is far from simple!
Binary Choice Item # 3 - flaw(s)? Goes against Commandment #4's statement that complicated sentence construction should be avoided AND against the writing guideline which states, Avoid items that include two concepts. The TWO concepts addressed in Binary Choice Item #3 are:A.) Assessment of students' attitudes have become more common. ANDB.) It's impossible to arrive at an accurate measurement-based inference.
Evidence: Binary Choice Item #3 appears to have two parts to it. We all agreed that two concepts are brought up in the question which would make it difficult for students to answer. As the powerpoint stated, our intenetion is not to TRICK the students. We just want to see what they know. An item like this, which brings up two concepts, makes things too complicated. Our whole group agreed that his test question seemed to contradict itself, and was not a simple question.
Binary Choice Item # 4 - flaw(s)? Item #4 appears to go against Commandment #4, Though shall not use complex syntax.To avoid constructing tests that fail to discriminate among test-takers, items should never be created that fail to elicit a reasonable amount of incorrect responses from students
Evidence: The above italicized statement is a very complex sentence. It does not provide the reader with a clear idea or concept. We all agreed that this statement has a complicated sentence construction, making it difficult to fully understand what is being asked. If you read it a couple of times, it finally starts to make sense, but by simply rewording it, we could really make this sentence a little bit clearer.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
*READ THE FOLLOWING INFORMATION AND THEN ANSWER THE TRUE/FALSE QUESTIONS.
115 students from Saint Xavier's School of Education were surveyed on what their ideal grade level to teach would be. Here are the results:
48 students selected Kindergarten thru 3rd Grade.
39 students selected 4th thru 5th grade.
28 students selected 6th thru 9th grade.
TRUE OR FALSE
1.) Less than half of the students surveyed selected kindergarten thru 3rd grade as their ideal grade level to teach.
2.) More SXU education students preferred to work with students in grades 6-9 than k-3.
3.)The least amount of SXU education students chose to work with students in grades 4-5.
4.)The majority SXU education students preferred to work with students in grades higher than the third grade.
(ANSWERS: 1. True, 2. False, 3. False, 4. True)
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)? Item #1 goes against the writing guideline which states, Avoid negatively stated stems AND the guideline which states, that the stem should consist of a self-contained question or problem.
Evidence: Multiple-Choice Item #1 ends with the word NOT. This negatively stated stem can confuse students and throw them off. Again, our goal here is not to trick our students. We just want to test their knowledge. If a negative word is thrown into a stem, it should be highlighted in some way so that it stands out from the rest of the question. This was not done in Multiple-Choice Item #1. Also, the writing guidelines state that lengthy stems and terse alternatives are preferred over short stems and long alternatives. Multiple-Choice Item #1 had a shorter stem and longer alternatives. The wordiness of the answers provided can confuse students and throw them off track.
Multiple-Choice Item # 2 - flaw(s)? Goes against Commandment #3, Thou shall not provide unintended clues.
Evidence: Multiple-Choice Item #2 ends like this, "...described as an:". That 'an' at the end of the question provides students with the clue that the answer should begin with a vowel. Answer A is the only answer that begins with a vowel, and therefore is the correct answer. A student doesn't have to know anything about the material in order to be able to answer this question. Therefore, question number 2 does not do a very good job testing the students' knowledge on the topic of interest.
Multiple-Choice Item # 3 - flaw(s)? Goes against Commandment #3, Thou shall not provided unintended clues AND the writing guideline which states, Use caution when considering the use of an 'ALL OF THE ABOVE' alternative answer.
Evidence: In Multiple-Choice #3, one of the alternatives is ALL OF THE ABOVE. The powerpoint cautions to consider using this type of an answer as it may provide students with unintended clues. All of us agreed that if a student knows two of the answers are correct, then they would know to select the ALL OF THE ABOVE answer. Therefore, this item does not properly and completely test the students on their content knowledge.
Multiple-Choice Item # 4 - flaw(s)? Goes against the writing guideline which states, Try to keep alternatives at about the same length.
Evidence: The alternatives provided in Multiple-Choice Item #4 may provide students with unintended clues, as choice D is significantly longer that the other choices. Students may believe that since the teacher put so many words into choice D, then it must be the answer. We all agreed in our discussion that as teachers, we should try keeping our alternative at about the same length. We don't want to clue students in to the right answer.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”** On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Column A
Column B
1. Bias-Detection
A. Determining DIF indices
2. Item Difficulty
B. Computing K-R formula
3. Reliability
C. Evaluating curricular alignment
4. Score Interpretation
D. Deriving percentiles
5. Validity
E. Calculating p// - values
Matching Item # 1 - flaw(s)?
We found several flaws with the matching item. One of the flaws is with the directions. The directions are very unclear which goes against Commandment 1. Another flaw we agreed on was “ordering responses logically”. This rule states that the responses should be in alphabetical order. The third flaw we saw was loading up on responses. The matching item has the same amount of responses as there are premises. One more flaw we noticed was the format. The matching item is too close together, it is not spaced out like it should be.
Evidence: As we stated above the directions are not clear which is a flaw that goes against commandment 1in writing guidelines. According to the guidelines it is important to always be clear in our directions for different tasks. The directions do not tell the student how many times an answer can be used. The second flaw was ordering responses . The responses in the item should be listed alphabetically which it fails to do. The reason the responses should be alphabetically is because if not the responses could give away clues or patterns. The other flaw was loading up on responses. The evidence is that there are exactly five premises and five responses. Since there is five premises that means there should be six or more responses. The reason is so students who know the majority of the information and answers can’t just guess right to get the answer. The last flaw is that the format is wrong. The format as you can see is too close together and very small.
1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)? Goes against Commandment #2, Though shall not employ ambiguous statements in your test item.
Evidence: Commandment 2: The word, 'they', appears in the questions. Due to the way the question is worded, we can't tell who 'they' is referring to, the administrators or the teachers?? This will affect how one may interpret the question, and therefore it flaws the question as a whole.
Binary Choice Item # 2 - flaw(s)? Goes against Commandment # 4, Though shall not employ complex syntax AND Commandment # 5, Though shall not use vocabulary that is too advanced for the audience.
Evidence: Binary Choice Item #2 uses extremely advanced and complex vocabulary that makes it difficult to comprehend what the question is even saying. Our whole group agreed that it was pretty much impossible to get through the whole question and actually understand what was being asked. This makes the question too difficult to answer. Commandment #4 also states to construct simple sentences. This question item is far from simple!
Binary Choice Item # 3 - flaw(s)? Goes against Commandment #4's statement that complicated sentence construction should be avoided AND against the writing guideline which states, Avoid items that include two concepts. The TWO concepts addressed in Binary Choice Item #3 are:A.) Assessment of students' attitudes have become more common. ANDB.) It's impossible to arrive at an accurate measurement-based inference.
Evidence: Binary Choice Item #3 appears to have two parts to it. We all agreed that two concepts are brought up in the question which would make it difficult for students to answer. As the powerpoint stated, our intenetion is not to TRICK the students. We just want to see what they know. An item like this, which brings up two concepts, makes things too complicated. Our whole group agreed that his test question seemed to contradict itself, and was not a simple question.
Binary Choice Item # 4 - flaw(s)? Item #4 appears to go against Commandment #4, Though shall not use complex syntax.To avoid constructing tests that fail to discriminate among test-takers, items should never be created that fail to elicit a reasonable amount of incorrect responses from students
Evidence: The above italicized statement is a very complex sentence. It does not provide the reader with a clear idea or concept. We all agreed that this statement has a complicated sentence construction, making it difficult to fully understand what is being asked. If you read it a couple of times, it finally starts to make sense, but by simply rewording it, we could really make this sentence a little bit clearer.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
*READ THE FOLLOWING INFORMATION AND THEN ANSWER THE TRUE/FALSE QUESTIONS.
115 students from Saint Xavier's School of Education were surveyed on what their ideal grade level to teach would be. Here are the results:
48 students selected Kindergarten thru 3rd Grade.
39 students selected 4th thru 5th grade.
28 students selected 6th thru 9th grade.
TRUE OR FALSE
1.) Less than half of the students surveyed selected kindergarten thru 3rd grade as their ideal grade level to teach.
2.) More SXU education students preferred to work with students in grades 6-9 than k-3.
3.)The least amount of SXU education students chose to work with students in grades 4-5.
4.)The majority SXU education students preferred to work with students in grades higher than the third grade.
(ANSWERS: 1. True, 2. False, 3. False, 4. True)
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)? Item #1 goes against the writing guideline which states, Avoid negatively stated stems AND the guideline which states, that the stem should consist of a self-contained question or problem.
Evidence: Multiple-Choice Item #1 ends with the word NOT. This negatively stated stem can confuse students and throw them off. Again, our goal here is not to trick our students. We just want to test their knowledge. If a negative word is thrown into a stem, it should be highlighted in some way so that it stands out from the rest of the question. This was not done in Multiple-Choice Item #1. Also, the writing guidelines state that lengthy stems and terse alternatives are preferred over short stems and long alternatives. Multiple-Choice Item #1 had a shorter stem and longer alternatives. The wordiness of the answers provided can confuse students and throw them off track.
Multiple-Choice Item # 2 - flaw(s)? Goes against Commandment #3, Thou shall not provide unintended clues.
Evidence: Multiple-Choice Item #2 ends like this, "...described as an:". That 'an' at the end of the question provides students with the clue that the answer should begin with a vowel. Answer A is the only answer that begins with a vowel, and therefore is the correct answer. A student doesn't have to know anything about the material in order to be able to answer this question. Therefore, question number 2 does not do a very good job testing the students' knowledge on the topic of interest.
Multiple-Choice Item # 3 - flaw(s)? Goes against Commandment #3, Thou shall not provided unintended clues AND the writing guideline which states, Use caution when considering the use of an 'ALL OF THE ABOVE' alternative answer.
Evidence: In Multiple-Choice #3, one of the alternatives is ALL OF THE ABOVE. The powerpoint cautions to consider using this type of an answer as it may provide students with unintended clues. All of us agreed that if a student knows two of the answers are correct, then they would know to select the ALL OF THE ABOVE answer. Therefore, this item does not properly and completely test the students on their content knowledge.
Multiple-Choice Item # 4 - flaw(s)? Goes against the writing guideline which states, Try to keep alternatives at about the same length.
Evidence: The alternatives provided in Multiple-Choice Item #4 may provide students with unintended clues, as choice D is significantly longer that the other choices. Students may believe that since the teacher put so many words into choice D, then it must be the answer. We all agreed in our discussion that as teachers, we should try keeping our alternative at about the same length. We don't want to clue students in to the right answer.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”**
On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Matching Item # 1 - flaw(s)?
We found several flaws with the matching item. One of the flaws is with the directions. The directions are very unclear which goes against Commandment 1. Another flaw we agreed on was “ordering responses logically”. This rule states that the responses should be in alphabetical order. The third flaw we saw was loading up on responses. The matching item has the same amount of responses as there are premises. One more flaw we noticed was the format. The matching item is too close together, it is not spaced out like it should be.
Evidence:
As we stated above the directions are not clear which is a flaw that goes against commandment 1in writing guidelines. According to the guidelines it is important to always be clear in our directions for different tasks. The directions do not tell the student how many times an answer can be used. The second flaw was ordering responses . The responses in the item should be listed alphabetically which it fails to do. The reason the responses should be alphabetically is because if not the responses could give away clues or patterns. The other flaw was loading up on responses. The evidence is that there are exactly five premises and five responses. Since there is five premises that means there should be six or more responses. The reason is so students who know the majority of the information and answers can’t just guess right to get the answer. The last flaw is that the format is wrong. The format as you can see is too close together and very small.